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Examiner's Remark 

Applicant's amendment, filed on 1/31/2009 has been received and entered into the file. 
The 101 rejections of claims 1-23 have been overcome in view of applicant's amendments and/or 
remarks and are hereby withdrawn. Claims 1-20 and 22-23 are amended and pending. Claims 
20 and 22 have been incorporated with allowable subject matter and is therefore allowable. 

Regarding applicant's argument that the amended limitation "applying a threshold mask 
to . . . provide a residual image that contains regions of said multi-channel image uniquely 
characterized by being potentially affected by a flash, but not uniquely characterized by being a 
potential facial region", the examiner would like to point out that instant application performs 
this step on Page 4, last paragraph and Page 5 first paragraph to obtain a flash mask which is 
later combined with other mask to represent the locations where the red-eye artifacts are most 
likely to occur. The teaching of Benati et al also discloses steps like this. Benati et al also apply 
a threshold mask the same way as the instant application (col. 4, lines 17-28) to obtain a bit 
map/mask. Although Benati et al apply thresholds for luminance, luma and saturation together, 
it is obvious that the step of applying the thresholds can be separately performed. Therefore the 
103 rejections have not been overcome. 

Specification 

1 . The specification is objected to as failing to provide proper antecedent basis for the 
claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the 
following is required. Claims 1-11 recite "applying a threshold mask to . . . provide a residual 
image that contains regions of said multi-channel image uniquely characterized by being 
potentially affected by a flash, but not uniquely characterized by being a potential facial region". 
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Claims 12-19 and 23 recite "provide a residual image that contains regions of said multi- 
channel image uniquely characterized by being potentially affected by a flash, but not uniquely 
characterized by a potential facial region". There is no antecedent basis for the claimed subject 
matter. 

Claim Objections 

2. Claims 1-19 and 23 are objected to because of the following informalities. Claims 1-11 
recite "applying a threshold mask to . . . provide a residual image that contains regions of said 
multi-channel image uniquely characterized by being potentially affected by a flash, but not 
uniquely characterized by being a potential facial region". Claims 12-19 and 23 recite "provide 
a residual image that contains regions of said multi-channel image uniquely characterized by 
being potentially affected by a flash, but not uniquely characterized by a potential facial region". 
There does not seem to be description on the residual image. Also what exactly is a threshold 
mask? Appropriate correction is required. For examination purposes, the examiner will interpret 
the claim limitation as described on Page 4 and 5 of the specification. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 12 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Benati 
et al (US Patent 5,748,764, cited by IDS). 
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Regarding claim 12, Benati et al teach a method to identify sub-regions of a multi-channel image 
containing red-eye (col. 4, lines 21-31), said multi-channel image having at least a first channel 
and a second channel (Fig. 5, first channel is hue and second channel is saturation, col. 4, lines 
231-31), said method comprising: 

(a) identifying a sub-region of said image as containing a red-eye region based upon, at 
least in part, applying a first mask to said first channel, said first mask comparing a first statistic 
of at least one pixel of said image to a first threshold (col. 4, lines 21-31, the first statistic is the 
hue value of a pixel, first threshold is either 700 or 1010, Fig. 6a, see also col. 3, lines 50-53, col. 
5, lines 17-27, col. 4, lines 18-27, col. 5, lines 43-67, col. 8, lines 40-67) to provide a residual 
image that contains regions of said multi-channel image uniquely characterized by being 
potentially affected by a flash, but not uniquely characterized by being a potential facial region 
(110 in Fig. 2, 210 in Fig. 3, Figs. 9-11, col. 4, lines 21-45, the bit map is used to provide such 
image in the same way as the instant application); and 

(b) applying a second mask to said second channel, said second mask comparing a 
second statistic of at least one pixel of said image to a second threshold, said second statistic 
being a different property than said first statistic (col. 4, lines 21-31, the second statistic is the 
saturation value of a pixel, the second threshold is either 65 or 256, the intei-pretation of statistic 
is as explained by the applicant in office action dated 7/7/2008, i.e., it can be pixel value, Fig. 
6C), 

(c) removing the identified said red eye from said multi-channel image (300 in Fig. 2). 
However they do not explicitly teach the second mask is applied to the residual image. But the 
steps of Benati et al on col. 4, lines 20-45 can be performed sequentially and the order does not 
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matter, that is, first applying the threshold for luminance first, then obtaining a residual image, 
and then applying the threshold for saturation to the residual image. It is mainly a design choice. 
It is desirable to be flexible when designing the algorithm. Therefore it would have been 
obvious to one of the ordinary skill in the art, at the time of invention, to apply the threshold for 
luminance first and provide a residual image and apply the threshold for saturation to the residual 
image. 

Regarding claim 14, Benati et al teach the method of claim 12 wherein said first threshold is 
different than said second threshold (col. 4, lines 21-31, the thresholds for hue and saturation are 
different). 

5. Claims 1-5, 7-11, 13, 15, 17 and 23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Benati et al (US Patent 5,748,764, cited by IDS) in view of Koga et al (U S 
Patent 5,848,185). 

Regarding claims 1, 7 and 23, Benati et al teach a method to identify sub-regions of a multi- 
channel image (Figs. 1 and 2) comprising: 

converting and providing said multi-channel image to a modified multi-channel image 
wherein at least one of said channels is an enhanced luminance channel that has more than 60% 
of the luminance information of said multi-channel image (col. 3, lines 46-62, col. 4, lines 17- 
28, col. 5, lines 43-57, col. 8, lines 44-63, the lightness channel contains 100% luminance and 
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therefore contains greater than 60% of the luminance information) and at least one of said 
channel is a saturation channel (col. 4, lines 17-28); and 

identifying a sub-region of said image as containing the red-eye region based upon, at 
least in part, processing said saturation channel by applying a saturation mask to one or more 
pixels of said image (col. 4, lines 17-45); 

removing the identified said red eye from said multi-channel image (300 in Fig. 2). 
However they do not explicitly teach the saturation mask is applied to the residual and said 
saturation mask comparing the standard deviation of the saturation value of a respective pixel to 
a threshold and they do not explicitly identifying location variations in said saturation based 
upon the standard deviation of the saturation value of pixels in said channel that substantially 
includes said saturation. 

But the steps of Benati et al on col. 4, lines 20-45 can be performed sequentially and the order 
does not matter, that is, first applying the threshold for luminance first, then obtaining a residual 
image, and then applying the threshold for saturation to the residual image. It is mainly a 
design choice. It is desirable to be flexible when designing the algorithm. Therefore it would 
have been obvious to one of the ordinary skill in the art, at the time of invention, to apply the 
threshold for luminance first and provide a residual image and apply the threshold for saturation 
to the residual image. 

In the same field of image segmentation and object detection, Koga et al teach applying a 
saturation mask to one or more pixels of said image, said saturation mask comparing the 
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standard deviation of the saturation value of a respective pixel to a threshold (col. 15, lines 34- 
41, the saturation mask compares the variance of the saturation value of a respective pixel to a 
threshold which is equivalent to comparing the standard deviation of the saturation value to a 
threshold because the variance is square of the standard deviation, Fig. 13, the respective pixel 
is a pixel in the image segment) to determine whether an image segment is monochromatic or 
color (Fig. 13). Koga et al also teach identifying location variations in said saturation based 
upon the standard deviation of the saturation value of pixels in said channel that substantially 
includes said saturation (col. 15, lines 22-41, the variance is the square of standard deviation, 
and variance of the image segment represents the location variations of standard deviation and 
variance in the image segment. Fig. 13). It is desirable to be efficient when detecting red-eye 
region by first focus on color region and detecting color region/segment. The method Koga et 
al is also a method to try in the method of Benati et al with predictable results (In re KSR v. 
Teleflex Inc). Therefore it would have been obvious to one of the ordinary skill in the art, at the 
time of invention, to use the method of Koga et al to detect whether an image segment is color 
or monochromatic first and then to find skin area and red eye area on the color area so that red- 
eye detection is more efficient. 

Regarding claim 2, Benati et al and Koga et al teach the method of claim 1 . Koga et al further 
teach wherein said standard deviation of said saturation value of a respective pixel is measured 
relative to the mean saturation of pixels in a neighborhood local to said respective pixel (col. 15, 
lines 21-34, the neighborhood is the image segment). 
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Regarding claims 3 and 8, Benati et al and Koga et al teach the method of claim 1 and claim 7. 
Benati et al further teach wherein said modified multi-channel image has hue, saturation, and 
intensity channels (col. 4, lines 17-28). 

Regarding claims 4 and 9, Benati et al and Koga et al teach the method of claim 3 and 8. Benati 
et al teach wherein said saturation channel represents the relative bandwidth of the visible output 
from a light source (col. 4, lines 17-28, the value of Sat by definition is the relative bandwidth of 
the visible output from a light source). 

Regarding claims 5 and 10, Benati et al and Koga et al teach the method of claim 4 and claim 9. 
Benati et al further teach wherein said hue is substantially the wavelength within the visible-light 
spectrum at which the energy output from a source is the greatest (col. 4, lines 17-28, inherent 
from the definition of hue). 

Regarding claim 11, Benati et al and Koga et al teach the method of claim 7. Benati et al further 
teach wherein each channel of said multi-channel image is processed differently to identify said 
sub-region of said image (col. 4, lines 17-28, the thresholds are different for different channels). 

Regarding claim 13, Benati et al teach the method of claim 12 where said first statistic is the 
intensity value of said pixel in said first channel. However they do not explicitly teach a second 
statistic is the standard deviation of a pixel in a second channel. In the field of object detection 
and extraction, Koga et al teach applying a saturation mask to one or more pixels of an image. 
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said saturation mask comparing the standard deviation of the saturation value of a respective 
pixel to a threshold (col. 15, lines 34-41, the saturation mask compares the variance of the 
saturation value of a respective pixel to a threshold which is equivalent to comparing the 
standard deviation of the saturation value to a threshold because the variance is square of the 
standard deviation, Fig. 13, the respective pixel is a pixel in the image segment). It is desirable to 
be efficient when detecting red-eye region by first focus on color region. Therefore it would 
have been obvious to one of the ordinary skill in the art, at the time of invention, to use the 
method of Koga et al in the method of Benati et al to detect and extract color image segment for 
red-eye detection so that more efficient detection and extraction can be achieved. 

Regarding claim 15, Benati et al and Koga et al teach the method of claim 13. Koga et al further 
teach wherein said standard deviation of said saturation value of a respective pixel is measured 
relative to the mean saturation of pixels in a neighborhood local to said respective pixel (col. 15, 
lines 21-34, the neighborhood is the image segment). 

Regarding claim 17, Benati et al and Koga et al teach the method of claim 13. Koga et al teach 
wherein said second channel represents saturation (col. 15, lines 34-41). 



6. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over Benati et al in 
view of Liang et al (US Patent 6,678,413 Bl). 
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Regarding claim 18, Benati et al teach the method of claim 17. However they do not teach the 
method comprising using a convex hull technique to identify contiguous regions. Liang et al 
teach a method comprising using a convex hull technique to identify contiguous regions when 
segmenting and identifying an object (col. 17, line 53-col. 18, line 6). It is desirable to 
represent and characterize an object by known techniques automatically (col. 3, lines 1-23 of 
Liang et al). Convex hull techniques are known to have the advantage of executing in linear 
time in a two-dimensional array as is usual in image processing. Therefore it would have been 
obvious to one of ordinary skill in the art, at the time of invention, to use the convex hull 
method of Liang et al to represent and identify contiguous regions in the method of Benati et al 
so that more efficient algorithm for red-eye detection can be developed. 

7. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Benati et al in 
view of Liang et al, further in view of Luo et al (US Patent 7,035,461). 

Regarding claim 19, Benati et al and Liang et al teach the method of claim 18. However they do 
not explicitly teach wherein contiguous regions having a size less than a threshold are removed 
as potential red-eye regions, said threshold computed dynamically based on the size of the input 
image. In the same field of endeavor, Luo et al teach resizing the input image (Fig. 12, col. 14, 
line 55-col. 15, line 11, col. 16, lines 46-59) and comparing the contiguous regions of the resized 
image with a threshold and removing the regions having a size less than a threshold (col. 16, 
lines 7-14, Figs. 12-13). Depending on the size of the input image, the size of the red-eye is 
different also. Scaling the input image dynamically based on the size of the input image and then 
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comparing the size of the contiguous regions with a threshold is equivalent to comparing the 
non-scaled contiguous region with a threshold that is dynamically computed based on the size of 
the input image. It is desirable to be efficient and correct when detecting red-eye pixels by 
eliminating pixels that are impossible to be red eyes (col. 1, lines 46-51 of Luo et al). Therefore 
it would have been obvious to one of ordinary skill in the art, at the time of invention, to use the 
method of Luo et al in the method of Benati and Liang et al so that contiguous pixels are 
eliminated/removed as non red-eye pixels depending on the size of input image. 

Allowable Subject Matter 

8. Claims 20 and 22 are allowed. Claims 6 and 16 are objected to as being dependent upon 
a rejected base claim, but would be allowable if rewritten in independent form including all of 
the limitations of the base claim and any intervening claims. A statement of reasons that claims 
6 and 16 cannot be rejected over the prior art is presented in the previous office action dated Oct. 
23, 2007 and will not be repeated here. A statement of reasons that claims 20 and 22 are 
allowable is provided in the office action dated 1/12/2009 and will not be repeated here. 

Conclusion 

9. THIS ACTION IS MADE FINAL. Apphcant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
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CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to YUZHEN GE whose telephone number is (571)272-7636. The 
examiner can normally be reached on 7:30am-4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella can be reached on 571-272-7778. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an apphcation may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Matthew C Bella/ Yuzhen Ge 

Supervisory Patent Examiner, Art Unit 2624 Examiner 

Art Unit 2624 



